fast_log
the fast log . This crate uses #! [forbid(unsafe_code)] to ensure everything is implemented in 100% Safe Rust.
A log implementation for extreme speed, using Crossbeam/channel ,once Batch write logs,fast log date, Appender architecture, appender per thread
-
Low overhead, log write based on thread, also support tokio/Future
-
High performance, use lockless message queue, log is stored in queue, then flush disk. It does not block the caller
-
Full APPEND mode file writing, high efficiency for solid state/mechanical disk (solid state and mechanical disk sequential write performance is better than random write)
-
When channel pressure increases, logs can be written in batches at a time
-
Built-in ZIP compression, compressed file name date + serial number, no need to worry about the log file is too large
-
Built-in log segmentation, custom log full number of immediately split
-
Built-in filtering configuration support, can be customized to filter out other library printed logs
-
Support custom compression algorithms, such as ZIP and LZ4
-
Support use
log::logger().flush()
method wait to flush disk -
Simple and efficient Appender architecture.Both configuration and customization are simple
-
Support custom file type, just like mmap/file
-----------------
log data-> | main channel(crossbeam) | ->
-----------------
---------------- ----------------------
-> |thread channel)| -> background thread | appender1 |
---------------- ----------------------
---------------- ----------------------
-> |thread channel)| -> background thread | appender2 |
---------------- ----------------------
---------------- ----------------------
-> |thread channel)| -> background thread | appender3 |
---------------- ----------------------
---------------- ----------------------
-> |thread channel)| -> background thread | appender4 |
---------------- ----------------------
-
How fast is?
-
no flush(chan_len=1000000) benches/log.rs
//MACOS(Apple M1MAX-32GB)
test bench_log ... bench: 85 ns/iter (+/- 1,800)
- all log flush into file(chan_len=1000000) example/bench_test_file.rs
//MACOS(Apple M1MAX-32GB)
test bench_log ... bench: 323 ns/iter (+/- 0)
- how to use?
= "0.4"
= { = "1.5"}
or enable zip/lz4/gzip Compression library
= "0.4"
# "lz4","zip","gzip"
= { = "1.5" , = ["lz4","zip","gzip"]}
Performance optimization(important)
- use
chan_len(Some(100000))
Preallocating channel memory reduces the overhead of memory allocation,for example:
use ;
Use Log(Console)
use ;
Use Log(Console Print)
use ;
Use Log(File)
use ;
use ;
Split Log(.log packer)
use RollingType;
use LogSize;
use LogPacker;
Split Log(mmap)
- Mmap (memory map) maps files or devices into memory and enables direct memory access.
- It is supported on Windows, Linux and macOS to map a file's contents into a program's address space.
- This allows very fast I/O without intermediate copying. Mmap is efficient for reading and writing files as it utilizes caching.
- It also enables easy sharing of memory between processes. Overall mmap provides fast, convenient file access while the operating system handles low level details like caching and swapping.
use Config;
use LogSize;
use MmapFile;
use RollingType;
use LogPacker;
Custom Log(impl do_log method)
use ;
use ;